FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction

Most cross-device federated learning (FL) studies focus on the model-homogeneous setting where the global server model and local client models are identical. However, such constraint not only excludes low-end clients who would otherwise make unique contributions to model training but also restrains clients from training large models due to on-device resource bottlenecks. In this work, we propose FedRolex, a partial training (PT)-based approach that enables model-heterogeneous FL and can train a global server model larger than the largest client model. At its core, FedRolex employs a rolling sub-model extraction scheme that allows different parts of the global server model to be evenly trained, which mitigates the client drift induced by the inconsistency between individual client models and server model architectures. We show that FedRolex outperforms state-of-the-art PT-based model-heterogeneous FL methods (e.g. Federated Dropout) and reduces the gap between model-heterogeneous and model-homogeneous FL, especially under the large-model large-dataset regime. In addition, we provide theoretical statistical analysis on its advantage over Federated Dropout and evaluate FedRolex on an emulated real-world device distribution to show that FedRolex can enhance the inclusiveness of FL and boost the performance of low-end devices that would otherwise not benefit from FL. Our code is available at: https://github.com/AIoT-MLSys-Lab/FedRolexComment: 20 pages, 7 Figures, Published in 36th Conference on Neural Information Processing And System

Optimizing the thermal performance of building envelopes for energy saving in underground office buildings in various climates of China

This article investigates the influence of the thermal performance of building envelopes on annual energy consumption in a ground-buried office building by means of the dynamic building energy simulation, aiming at offering reasonable guidelines for the energy efficient design of envelopes for underground office buildings in China. In this study, the accuracy of dealing with the thermal process for underground buildings by using the Designer's Energy Simulation Tool (DeST) is validated by measured data. The analyzed results show that the annual energy consumptions for this type of buildings vary significantly, and it is based on the value of the overall heat transfer coefficient (U-value) of the envelopes. Thus, it is necessary to optimize the U-value for underground buildings located in various climatic zones in China. With respect to the roof, an improvement in its thermal performance is significantly beneficial to the underground office building in terms of annual energy demand. With respect to the external walls, the optimized U-values completely change with the distribution of the climate zones. The recommended optimal values for various climate zones of China are also specified as design references for public office building in underground in terms of the building energy efficiency

RecUP-FL: Reconciling Utility and Privacy in Federated Learning via User-configurable Privacy Defense

Federated learning (FL) provides a variety of privacy advantages by allowing clients to collaboratively train a model without sharing their private data. However, recent studies have shown that private information can still be leaked through shared gradients. To further minimize the risk of privacy leakage, existing defenses usually require clients to locally modify their gradients (e.g., differential privacy) prior to sharing with the server. While these approaches are effective in certain cases, they regard the entire data as a single entity to protect, which usually comes at a large cost in model utility. In this paper, we seek to reconcile utility and privacy in FL by proposing a user-configurable privacy defense, RecUP-FL, that can better focus on the user-specified sensitive attributes while obtaining significant improvements in utility over traditional defenses. Moreover, we observe that existing inference attacks often rely on a machine learning model to extract the private information (e.g., attributes). We thus formulate such a privacy defense as an adversarial learning problem, where RecUP-FL generates slight perturbations that can be added to the gradients before sharing to fool adversary models. To improve the transferability to un-queryable black-box adversary models, inspired by the idea of meta-learning, RecUP-FL forms a model zoo containing a set of substitute models and iteratively alternates between simulations of the white-box and the black-box adversarial attack scenarios to generate perturbations. Extensive experiments on four datasets under various adversarial settings (both attribute inference attack and data reconstruction attack) show that RecUP-FL can meet user-specified privacy constraints over the sensitive attributes while significantly improving the model utility compared with state-of-the-art privacy defenses

TransMUSE: Transferable Traffic Prediction in MUlti-Service Edge Networks

The Covid-19 pandemic has forced the workforce to switch to working from home, which has put significant burdens on the management of broadband networks and called for intelligent service-by-service resource optimization at the network edge. In this context, network traffic prediction is crucial for operators to provide reliable connectivity across large geographic regions. Although recent advances in neural network design have demonstrated potential to effectively tackle forecasting, in this work we reveal based on real-world measurements that network traffic across different regions differs widely. As a result, models trained on historical traffic data observed in one region can hardly serve in making accurate predictions in other areas. Training bespoke models for different regions is tempting, but that approach bears significant measurement overhead, is computationally expensive, and does not scale. Therefore, in this paper we propose TransMUSE, a novel deep learning framework that clusters similar services, groups edge-nodes into cohorts by traffic feature similarity, and employs a Transformer-based Multi-service Traffic Prediction Network (TMTPN), which can be directly transferred within a cohort without any customization. We demonstrate that TransMUSE exhibits imperceptible performance degradation in terms of mean absolute error (MAE) when forecasting traffic, compared with settings where a model is trained for each individual edge node. Moreover, our proposed TMTPN architecture outperforms the state-of-the-art, achieving up to 43.21% lower MAE in the multi-service traffic prediction task. To the best of our knowledge, this is the first work that jointly employs model transfer and multi-service traffic prediction to reduce measurement overhead, while providing fine-grained accurate demand forecasts for edge services provisioning

Liquid-phase Hydrogenation of Phenol to Cyclohexanone over Supported Palladium Catalysts

The ZSM-5, g-Al2O3, SiO2 and MgO supported Pd-catalysts were prepared for the phenol hydrogenation to cyclohexanone in liquid-phase. The natures of these catalysts were characterized by XRD, N2 adsorption-desorption analysis, H2-TPR, CO2-TPD and NH3-TPD. The catalytic performance of the supported Pd-catalyst for phenol hydrogenation to cyclohexanone is closely related to nature of the support and the size of Pd nanoparticles. The Pd/MgO catalyst which possesses higher basicity shows higher cyclohexanone selectivity, but lower phenol conversion owing to the lower specific surface area. The Pd/SiO2 catalyst prepared by precipitation gives higher cyclohexanone selectivity and phenol conversion, due to the moderate amount of Lewis acidic sites, and the smaller size and higher dispersion of Pd nanoparticles on the surface. Under the reaction temperature of 135 oC and H2 pressure of 1 MPa, after reacting for 3.5 h, the phenol conversion of 71.62% and the cyclohexanone selectivity of 90.77% can be obtained over 0.5 wt% Pd/SiO2 catalyst. Copyright © 2016 BCREC GROUP. All rights reservedReceived: 7th March 2016; Revised: 13rd May 2016; Accepted: 7th June 2016How to Cite: Fan, L., Zhang, L., Shen, Y., Liu, D., Wahab, N., Hasan, M.M. (2016). Liquid-phase Hydrogenation of Phenol to Cyclohexanone over Supported Palladium Catalysts. Bulletin of Chemical Reaction Engineering & Catalysis, 11 (3): 354-362 (doi: 10.9767/bcrec.11.3.575.354-362)Permalink/DOI: http://doi.org/10.9767/bcrec.11.3.575.354-36